Exploring Immersive Technologies in Learning. J.UCS Special Issue

Immersive technologies have presented very promising opportunities for innovation in learning. In recent years, they have become more accessible and we are just beginning to see an increase in their adoption, addressing user needs in various domain, particularly in education. At the time of this writing (second quarter, 2020) and given the global emergency situation that COVID-19 has imposed to the world, it is more relevant than ever to look at ways to materialise the promise that they represent, tackling some of the bigger challenges that the society face. Therefore, we, the community of scholars and practitioners at the intersection of learning, technology and game sciences, find ourselves at the centre of a massive opportunity to demonstrate the potential that Immersive Learning can bring to support society in this challenging times, as the world continues to deal with a global health crisis

Using Serious Games for Learning British Sign Language Combining Video, Enhanced Interactivity, and VR Technology

One in every six persons in the UK suffers a hearing loss, either as a condition they have been born with, or they developed during their life. Nine hundred thousand people in the UK are severely or profoundly deaf. Based on a study by Action on Hearing Loss UK in 2013 only 17 percent of this population, can use the British Sign Language (BSL). That leaves a massive proportion of people with a hearing impediment who do not use sign language struggling in social interaction and suffering from emotional distress. It also leaves even a larger proportion of Hearing people who cannot communicate with those of the deaf community. This paper presents a Serious Game (SG) that aims to close the communication gap between able hearing people and people with hearing impairment by providing a tool that facilitates BSL learning targeting the adult population. The paper presents the theoretical framework supporting adult learning based on which a SG game using Virtual Reality (VR) technology has been developed. The paper explains the experimental framework of the study. It presents the creation of the research instruments to facilitate the study comprising of a SG that integrates video and conventional video-based educational material. It reports and analyses the study results that demonstrate the advantage of the SG in effectively supporting users learning a set of BSL signs. It also presents qualitative outcomes that inform the further development of the game to serve learning needs. The paper closes with conclusions, directions for further development of this educational resource, and future studies

Adult Learning Sign Language by combining video, interactivity and play

One in every six persons in the UK suffers a hearing loss, either as a condition they have been born with or a disorder they acquired during their life. 900,000 people in the UK are severely or profoundly deaf and based on a study by Action On Hearing Loss UK in 2013 only 17 percent of this population, can use the British Sign Language (BSL). That leaves a massive proportion of people with a hearing impediment who do not use sign language struggling in social interaction and suffering from emotional distress, and an even larger proportion of Hearing people who cannot communicate with those of the deaf community. This paper presents a theoretical framework for the design of interactive games to support learning BSL supporting the entire learning cycle, instruction, practice and assessment. It then describes the proposed design of a game based on this framework aiming to close the communication gap between able hearing people and people with a hearing impediment, by providing a tool that facilitates BSL learning targeting adult population. The paper concludes with the planning of a large scale study and directions for further development of this educational resource

3D Hand Pose Estimation with Neural Networks

We propose the design of a real-time system to recognize and interprethand gestures. The acquisition devices are low cost 3D sensors. 3D hand pose will be segmented, characterized and track using growing neural gas (GNG) structure. The capacity of the system to obtain information with a high degree of freedom allows the encoding of many gestures and a very accurate motion capture. The use of hand pose models combined with motion information provide with GNG permits to deal with the problem of the hand motion representation. A natural interface applied to a virtual mirrorwriting system and to a system to estimate hand pose will be designed to demonstrate the validity of the system

Perceptual User Interface Framework for Immersive Information Retrieval Environments (An Experimental Framework for Testing and Rapid Iteration)

The use of perceptual inputs is an emerging area within HCI that suggests a developing Perceptual User Interface (PUI) that may prove advantageous for those involved in mobile serious games and immersive social network environments. Since there are a large variety of input devices, software platforms, possible interactions, and myriad ways to combine all of the above elements in pursuit of a PUI, we propose in this paper a basic experimental framework that will be able to standardize study of the wide range of interactive applications for testing efficacy in learning or information retrieval and also suggest improvements to emerging PUIs by enabling quick iteration. This rapid iteration will start to define a targeted range of interactions that will be intuitive and comfortable as perceptual inputs, and enhance learning and information retention in comparison to traditional GUI systems. The work focuses on the planning of the technical development of two scenarios, and the first steps in developing a framework to evaluate these and other PUIs for efficacy and pedagogy

Football Video Annotation Based on Player Motion Recognition Using Enhanced Entropy

This paper presents a semi-parametric Algorithm for parsing football video structures. The approach works on a two interleaved based process that closely collaborate towards a common goal. The core part of the proposed method focus perform a fast automatic football video annotation by looking at the enhance entropy variance within a series of shot frames. The entropy is extracted on the Hue parameter from the HSV color system, not as a global feature but in spatial domain to identify regions within a shot that will characterize a certain activity within the shot period. The second part of the algorithm works towards the identification of dominant color regions that could represent players and playfield for further activity recognition. Experimental Results shows that the proposed football video segmentation algorithm performs with high accuracy

Work-In-Progress—CrimOPS—Gamified Virtual Simulations for Authentic Assessment in Criminology

Authentic assessment is the concept of using creative learning experiences to assess students’ skills and knowledge in conditions that simulate reality. The importance of authentic assessment in higher education is recognized as it helps measure student performance and attainment of skills related to the expected learning outcomes of the degree program, they study that map the skills required in their profession in the related industry. There is also growing evidence that authentic assessment can enhance student engagement and improve the student experience and learning outcomes. VR coupled with gamification offers the authentic context for students to apply their knowledge and skills while solving real-world problems. This paper presents a work-in-progress study that focuses on the creation of an educational resource that effectively supports teaching and learning crime-scene analysis and evidenced-based inferences in Criminology through simulated crime scenes. The paper describes a participatory design process involving students and academics as co-creators of a VR game, the CrimOPS, it presents the initial prototype derived from this collaboration and preliminary study results

Work-in-progress-Gamifying the process of Learning Sign Language in VR

This study recognizes the need for providing a tool to facilitate learning British Sign Language (BSL). Virtual reality coupled with gamification holds exciting prospects to accommodate this need. This paper presents a work-in-progress study that focuses on evaluating the impact of combining scaffolded instruction with gamification to design a 3D interactive game to support learning the BSL alphabet. The paper outlines the project motivations, its aims and objectives, the proposed research methodology, and the expected contributions to knowledge

Model Probability in Self-organising Maps

Growing models have been widely used for clustering or topology learning. Traditionally these models work on stationary environments, grow incrementally and adapt their nodes to a given distribution based on global parameters. In this paper, we present an enhanced unsupervised self-organising network for the modelling of visual objects. We first develop a framework for building non-rigid shapes using the growth mechanism of the self-organising maps, and then we define an optimal number of nodes without overfitting or underfitting the network based on the knowledge obtained from information-theoretic considerations. We present experimental results for hands and we quantitatively evaluate the matching capabilities of the proposed method with the topographic product